Response to office action of 5/17/07 for appl. 10/780,500 

Remarks 

Claims 1-3, 5-28 and 32-37 are pending in this application. Claims 1-3, 
5-10 and 32-35 stand rejected under 35 U.S.C. 103(a) as being unpatentable 
over Szeliski et al. US patent no. 6,157,747 in view of Luken US patent no. 
5,923,334 and further in view of Katayama et al. US Pub. no. 2002/0081019 
Al. Claims 11, 12, 22-28 and 36 stand rejected under 35 U.S.C. 103(a) as 
being unpatentable over Szeliski et al. US patent no. 6,157,747 in view of 
Katayama et al. US Pub. no. 2002/0081019 Al. Claims 13-21 and 37 stand 
rejected under 35 U.S.C. 103(a) as being unpatentable over Szeliski et al. US 
patent no. 6,157,747 view of Katayama et al. US Pub. no. 2002/0081019 Al 
and further in view of Blank US patent no. 5,469,536. 

The Applicant appreciates the opportunity for the telephonic interview 
held between Applicant's attorney, John Conway, and the Examiner on August 
21, 2007. The Katayama reference was discussed as combined with the 
Szeliski and Luken references. Applicant's attorney pointed out that 
Katayama's method relies on exact camera posture information associated with 
each of a series of images of an object to determine the shape of a 3D object 
and that such information is not available from the mosaiced image generated 
by the combination of Szeliski's method and Luken's method. Thus, the 
methods of these references are not combinable to achieve the embodiment of 
Claim 1 . No agreement was reached during the interview. 

Claims 1, 5, 12, 18, 22, and 36 have been amended to correct 
typographical errors. No new matter has been added. 
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Claim Rejections - 35 U.S.C § 103(a) 

A. Claims 1-3, 5-10 and 32-35 stand rejected under 35 U.S.C. 103(a) as 

being unpatentable over Szeliski et al., US patent no. 6,157,747, in view of 

Luken, US patent no. 5,923,334, and further in view of Katayama et al., US 

Pub. no. 2002/0081019 Al. 

The rejection of Claims 1-3, 5-10 and 32-35 under 35 U.S.C. 103(a) as 

being unpatentable over Szeliski et al., US patent no. 6,157,747, in view of 

Luken, US patent no. 5,923,334, and further in view of Katayama et al., US 

Pub, no. 2002/0081019 Al, lacks a prima facie case of obviousness because 

there is no reasonable expectation of success in combining the teachings of 

these three references. 

Claim 1 of the subject application requires (in part): 

"...creating a three dimensional model of the visual scene from the 

transformed and aligned image panoramas using the reference coordinate 

system, wherein creating a three dimensional model includes identifying a 

selected object in the transformed and aligned image panoramas and 

associating geometry information with the selected object, the geometry 

information comprising 3-D coordinates describing the position and orientation 

of the selected object in the reference coordinate system." 

The Office Action admits that: 

"Szeliski and Luken do not directly teach a method wherein creating a three 
dimensional model includes identifying a selected object in the transformed 
and aligned image panoramas and associating geometry information with the 
selected object, the geometry information comprising 3-D coordinates 
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describing the position and orientation of the selected object in the reference 
coordinate system." 

See, Office Action, page 3, para. 4. 

On the other hand, Katayama '019 teaches a method for extracting 3D 

shape information from a series of images of a 3D object. Katayama's 

apparatus for acquiring the images on which to perform the extraction is 

shown in fig. 9 of Katayama '019, for an embodiment where the images are 

acquired using a single camera system. (Katayama '019's embodiment that 

uses a dual camera system, as shown in fig. 2, is discussed below on page 16 

of this response.) 
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FIG, 9 




X 



Katayama's system, shown in fig. 9, uses the marks "A", "B", "C", and "D" 
printed on a background paper 703 to establish the camera posture (i. e., 
position and orientation of the camera 700). (See, Katayama '019, paras. 159 
and 163). The camera is moved in the direction of the arrow and a series of 
images are taken. (See, fig. 11, steps s25 to s28.). Katayama then extracts the 
3D coordinates of the object 705 (Katayama '019, paras. 169) from the images 
and the posture of the camera , as determined for each image. Katayama's 
method requires determination of the exact camera posture, from the marks 
(i.e., "A", "B", "C", and "D") on the background paper. (Katayama '019, para. 
156.) Without these reference marks in the background of the image, 
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Katayama's method will not work, since step S26, "detect posture" cannot be 
performed. Thus, Katayama's method is not combinable with the methods of 
Szeliski and Luken because neither Szeliski's nor Luken's method provides 
images with the alignment marks that are needed to determine the camera 
posture for Katayama's extraction method. Please note that exact camera 
posture determination for the acquired image is but one of the prerequisite's for 
Katayama's method of extracting 3D shape information that are not satisfied by 
Szeliski or Luken's methods. Other prerequisites not satisfied by Szeliski's or 
Luken's methods include adjustment of camera parameters and acquiring a 
series of images from different camera postures (Katayama '019, paras. 143 to 
149.) 

To establish prima facie obviousness of a claimed invention, there must 
be a reasonable expectation of success in combining the teachings. (See, e.g., 
MPEP 2143). As shown above, no reasonable expectation of success exists in 
combining Katayama's method of object extraction with the methods of Szeliski 
and Luken, because Szeliski's and Luken's images do not allow Katayama's 
camera posture detection step to be performed successfully. Therefore, Claim 
1 is deemed non-obvious over Szeliski '747, Luken '334 and Katayama '019 
because a prima facie case of obviousness is lacking. Claims 2-3 and 5-10 
depend from Claim 1 and add further limitations. These Claims are deemed 
non-obvious over Szeliski '747, Luken '334 and Katayama '019 for at least the 
same reasons as Claim 1. 
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Claim 32 is deemed non-obvious over Szeliski '747, Luken '334 and 
Katayama '019 for the same reasons as for Claim 1 because Claim 32 contains 
claim limitations analogous to the claim limitations cited above for Claim 1 . 
Claims 33-35, which depend from Claim 32 and add further limitations, are 
deemed non-obvious over Szeliski '747 in view of Luken '334 and Katayama 
'019 for at least the same reasons as for Claim 32. 

B. Claims 11, 12, 22-28 and 36 stand rejected under 35 U.S.C. 103(a) as 
being unpatentable over Szeliski et al., US patent no. 6,157,747, in view of 
Katayama et al., US Pub. no. 2002/0081019 Al. 

The rejection of Claims 11, 12, 22-28 and 36 as amended, under 35 
U.S.C. 103(a) as being unpatentable over Szeliski et al. US patent no. 
6,157,747 in view of Katayama et al. US Pub, no. 2002/0081019 Al lacks a 
prima facie case of obviousness because there is no reasonable expectation of 
success in combining the teachings of these references. 

Claims 11 requires (in part). 

"...identifying a selected object in the image panorama and associating 

geometry information with the selected object, the geometry information 

comprising 3-D coordinates describing the position and orientation of the 

selected object in a reference coordinate system;.." 

The Office Action admits on page 7, paragraph 2 that: 

"Szeliski does not directly teach a method wherein creating a 
three dimensional model includes identifying a selected 
object in the image panorama and associating geometry 
information with the selected object, the geometry 
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information comprising 3-D coordinates describing the 
position and orientation of the selected object in the 
reference coordinate system." 

As shown above for Claim 1 , no reasonable expectation of success exists 
in combining Katayama '019's method of object extraction with the method of 
Szeliski because Szeliski's images do not allow Katayama's camera posture 
detection step to be performed successfully. Thus, a prima facie case of 
obviousness is lacking for the combination of Szeliski '747 with Katayama '019. 
Therefore, Claim 1 1 is deemed non-obvious over Szeliski '747 and Katayama 
'019. Because Claim 12 depends from Claim 1 1 and adds further limitations, 
Claim 12 is also deemed non-obvious over Szeliski '747 and Katayama '019 for 
at least the same reasons as for Claim 1 1 . 

Further, Claims 22 and 36 are deemed non-obvious over Szeliski '747 and 
Katayama '019 for the same reasons as for Claim 11, because Claims 22 and 
36 contains claim limitations analogous to the claim limitations cited above for 
Claim 11. Claims 23-28, which depend from Claim 22 and add further 
limitations, are deemed non-obvious over Szeliski '747 in view of Katayama 
'019 for at least the same reasons as for Claim 22. 

C. Claims 13-21 and 37 stand rejected under 35 U.S.C. 103(a) as being 
unpatentable over Szeliski et al. US patent no. 6,157,747 view of Katayama et 
al. US Pub. no. 2002/0081019 Al and further in view of Blank, US patent no. 
5,469,536. 
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The rejection of Claims 13-21 and 37 under 35 U.S.C. 103(a) as being 
obvious over Szeliski '747 in view of Katayama '019 and Blank '536 lacks a 
prima facie case of obviousness because there is no reasonable expectation of 
success in combining the teachings of these references. 

The rejections of Claims 13-21 and 37 for obviousness rely on Szeliski 
'747 and Katayama '019 for teaching the limitations of Claims 1 1 and 36 from 
which these claim depend, respectively. As shown above, the combination of 
Szeliski '747 and Katayama '019 do not yield a prima facie case of obviousness 
because there is no reasonable expectation of success in combining these 
references to achieve the cited limitations of Claims 1 1 and 36. Further, Blank 
'536 provides no teaching that increases the expectation of success in 
combining the teachings of Szeliski and Katayama in achieving the cited 
limitations of claims 1 1 and 36. Because Claims 13-21 and 37 depend from 
Claims 1 1 and 36, respectively, and add further limitations, Claims 13-21 and 
37 are deemed non-obvious over Szeliski '747 in view of Katayama '019 and 
Blank '536 for at least the same reasons as for Claim 1 1 and 36, respectively. 
Katayama '019's Dual Camera Embodiment 

Fig. 2 of Katayama '019 shows the dual camera embodiment of 
Katayama 's object extractor. 
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FIGL 2 




Katayama's method depends on determination of the camera pose (i.e,, 
position in space and orientation) for each of the images taken by the dual 
image capture camera 1 . Katayama uses a hardware-based posture detector 
(labeled "201" in fig. 3 of Katayama '019 and not shown in fig. 2) to determine 
the camera pose in this embodiment. (See, Katayama '019, para. 88.) Without 
the information provided by the posture detector, Katayama's method of object 
extraction fails because camera posture must be combined with the image 
signals from sensors 100L and 100R to calculate object coordinates. (See, 
Katayama '019 step "SI 7" in fig. 5B and para. 150.) Because neither Szeliski's 
nor Luken's images provide such posture information for the images, there is 
no expectation of success in combining the teachings of Szeliski, Luken and 
Katayama. This follows because a key ingredient, camera posture, is missing 
which is needed to perform Katayama's method of object extraction. 
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Applicant requests reconsideration of all pending claims and a notice of 
allowance. The Examiner is requested to telephone the undersigned if any 
matters remain outstanding so that they may be resolved expeditiously. The 
Commissioner is hereby authorized to charge any deficiency in the fees filed, 
asserted to be filed or which should have been filed herewith to our Deposit 
Account No. 19-4972. 

Respectfully submitted, 
/John L. Conway, #48,241/ 
John L. Conway 

Bromberg & Sunstein LLP Registration No. 48,241 

125 Summer Street Attorney for Applicant 

Boston, MA 02110-1618 
(617) 443-9292 

02894/00108 672339.2 
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